Repository navigation
feat(symbols,watch): Stage 3 — tree-sitter WASM symbols, per-symbol chunks, watchRoots, structured search - #13
Merged
Merged
Conversation
…l chunks, watchRoots, structured search - extractSymbolsTree/symbolsFor: swift/ts/tsx/js/py via web-tree-sitter@0.25.4 + tree-sitter-wasms pack; caller-owned grammar map; regex fallback unchanged. Vendored 0.1.8 swift wasm — pack swift >= 0.1.10 is process-fatal on Node 24 (V8 turboshaft Zone OOM in background tier-up; details in grammars/README.md). - NormalizedDocument.chunks? passthrough (F1): pre-chunked docs stored verbatim; repoConnector emits one chunk per symbol (heading_path "symbol: name (kind)"). - repoDocumentFor (F3): per-file normalize for watcher batches, same skip rules. - watchRoots: debounced batches, one OS watch per root, ignore policy, healthy(). - EsCompatRetriever.search (F2): BM25 + heading_path prefix filter, throws on error. - Suite 226 passed (WASM tests run WITHOUT --liftoff-only — vendored swift clean). v0.8.0
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What (Lionidas Stage 3 Part A — mechanism only, no Lionidas words)
extractSymbolsTree/symbolsForover swift/typescript/tsx/js/python; grammar policy is caller-supplied (SymbolGrammars, ext → pack name OR absolute .wasm path);defaultSymbolGrammars()preset; v1 regex fallback preserved.grammars/tree-sitter-swift.wasm(from tree-sitter-wasms@0.1.8): the pack swift ≥ 0.1.10 is process-fatal on Node ≥ 24 —Fatal process out of memory: Zonein V8 turboshaft background wasm tier-up (parse succeeds, process dies later;--liftoff-onlyavoids it but is banned in NODE_OPTIONS). Isolated empirically: only swift affected, 0.1.8 clean. Full story + re-extract instructions ingrammars/README.md.NormalizedDocument.chunks?passthrough, ingest stores verbatim; repoConnector now emits one chunk per symbol (heading_path: "symbol: <name> (<kind>)"), doc content stays a browsable summary.repoDocumentFor(cfg, absPath)(F3) — per-file normalize honoring skip rules; freshness watcher path re-ingests one file, no walk.watchRoots— debounced changed-path batches, one OS watch per root, ignore policy,healthy()/describe(); errors mark the root dead, never throw.EsCompatRetriever.search({query, headingPrefix?, size})(F2) — BM25 + heading_path prefix filter; same throw contract asretrieve().Verify
--liftoff-only, proving the vendored swift is clean in-process)web-tree-sitter+node:moduleadded (declared deps)Consumer wiring (Lionidas) pins this PR sha and follows after merge.